Skip to content

Add LangChain invoke workflow and invoke agent span support#4449

Closed
wrisa wants to merge 47 commits into
open-telemetry:mainfrom
wrisa:langchain-workflow-type
Closed

Add LangChain invoke workflow and invoke agent span support#4449
wrisa wants to merge 47 commits into
open-telemetry:mainfrom
wrisa:langchain-workflow-type

Conversation

@wrisa

@wrisa wrisa commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Description

  • Introduces support for tracing LangChain workflow-level operations (first chain) and agent invocation.
  • Addson_chain_start, on_chain_start and on_chain_error for tracking workflow invocation and agent invocation alongside the existing LLM inference spans.
  • Includes a agent without workflow example (examples/agent/main.py)
  • Includes a agent with workflow example (examples/agent/single_node_agent.py)
  • Includes a workflow without agent example (examples/workflow/main.py)
  • Comprehensive tests covering chain execution tracing.

Other changes:

See workflow and inference spans:
Screenshot 2026-04-16 at 9 38 54 AM

See agent and inference spans:
Screenshot 2026-05-04 at 2 09 58 PM

See workflow, agent and inference spans:
Screenshot 2026-05-04 at 2 13 49 PM

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Test A

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@wrisa wrisa changed the title Add workflow and refactor LLM for langchain Add LangChain workflow span support and refactor LLM invocation Apr 16, 2026
@wrisa
wrisa marked this pull request as ready for review April 23, 2026 16:45
@wrisa
wrisa requested a review from a team as a code owner April 23, 2026 16:45
@lmolkova
lmolkova requested a review from Copilot April 23, 2026 17:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds workflow-level tracing for LangChain (top-level chain runs) and refactors LLM span creation to use the newer GenAI invocation APIs, including an example and expanded test coverage.

Changes:

  • Add workflow invocation (INTERNAL span) start/stop/error handling via on_chain_* callbacks.
  • Refactor LLM spans to use InferenceInvocation (start_inference(), stop(), fail()), and introduce workflow invocation tracking.
  • Add opentelemetry-util-genai dependency plus a LangGraph workflow example and new tests.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
instrumentation-genai/opentelemetry-instrumentation-langchain/tests/test_workflow_chain.py Adds unit tests validating workflow span creation, CSA propagation, and error/no-op paths.
instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/invocation_manager.py Makes invocation state nullable to support runs without an associated GenAI invocation.
instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/callback_handler.py Implements workflow spans for chains and migrates LLM handling to InferenceInvocation.
instrumentation-genai/opentelemetry-instrumentation-langchain/pyproject.toml Updates core instrumentation dependency and adds explicit opentelemetry-util-genai dependency.
instrumentation-genai/opentelemetry-instrumentation-langchain/examples/workflow/requirements.txt Adds dependencies for the new workflow example.
instrumentation-genai/opentelemetry-instrumentation-langchain/examples/workflow/main.py Adds a LangGraph StateGraph workflow example that invokes an LLM node.
instrumentation-genai/opentelemetry-instrumentation-langchain/CHANGELOG.md Notes the new workflow span support/refactor in the changelog.

@github-project-automation github-project-automation Bot moved this to Approved PRs in Python PR digest May 7, 2026
)
return LANGGRAPH_IDENTIFIER in name or LANGGRAPH_IDENTIFIER in graph_id

return True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, if a top level chain has no serialized data, should it be classified as a workflow?

@wrisa wrisa May 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback return True is intentional — it handles the case where LangGraph chain doesn't populate serialized but is still the root invocation. Without this, top-level chains with missing serialization would be silently suppressed, producing no span at all for the outermost operation.
I can add clarification comment that would make the intent explicit ?

      # No serialized data to inspect, but this is a top-level chain
      # (parent_run_id is None). Treat it as a workflow so the outermost
      # invocation always gets a span, even if the chain didn't serialize itself.
      return True

@lmolkova lmolkova left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run the example, I see

Image

Not sure if we should expect to see invoke_agent somewhere?

I've tried modifying it to be more complicated

example ```python def build_graph(llm: ChatOpenAI): """Draft → critique → (revise | END) workflow over ``GraphState``."""
def draft_node(state: GraphState) -> dict[str, str]:
    response = llm.invoke(
        [
            SystemMessage(
                content=(
                    "You are a concise technical writer. "
                    "Write a short first-pass response (3-5 sentences)."
                )
            ),
            HumanMessage(content=state["request"]),
        ]
    )
    return {"draft": response.content}

def critique_node(state: GraphState) -> dict[str, str]:
    response = llm.invoke(
        [
            SystemMessage(
                content=(
                    "You are a strict editor. Review the draft below for "
                    "factual accuracy, clarity, and tone.\n"
                    f"If the draft is acceptable as-is, reply with just '{APPROVAL_TOKEN}'.\n"
                    "Otherwise, list the specific issues to fix in 1-3 bullet points."
                )
            ),
            HumanMessage(
                content=(
                    f"Original request:\n{state['request']}\n\n"
                    f"Draft:\n{state['draft']}"
                )
            ),
        ]
    )
    return {"critique": str(response.content)}

def route_after_critique(
    state: GraphState,
) -> Literal["revise", "finalize"]:
    critique = state["critique"].strip()
    return "finalize" if critique.startswith(APPROVAL_TOKEN) else "revise"

def revise_node(state: GraphState) -> dict:
    response = llm.invoke(
        [
            SystemMessage(
                content=(
                    "You are the same technical writer. Rewrite your "
                    "draft to address every issue raised by the editor. "
                    "Keep the response concise."
                )
            ),
            HumanMessage(
                content=(
                    f"Original request:\n{state['request']}\n\n"
                    f"Your draft:\n{state['draft']}\n\n"
                    f"Editor's critique:\n{state['critique']}"
                )
            ),
        ]
    )
    return {"final": response.content}

def finalize_node(state: GraphState) -> dict:
    return {"final": state["draft"]}

builder = StateGraph(GraphState)
builder.add_node("draft", draft_node)
builder.add_node("critique", critique_node)
builder.add_node("revise", revise_node)
builder.add_node("finalize", finalize_node)

builder.add_edge(START, "draft")
builder.add_edge("draft", "critique")
builder.add_conditional_edges(
    "critique",
    route_after_critique,
    {"revise": "revise", "finalize": "finalize"},
)
builder.add_edge("revise", END)
builder.add_edge("finalize", END)

return builder.compile()
</details>

Here's what I see from this branch:

<img width="888" height="180" alt="Image" src="https://github.com/user-attachments/assets/ecdd0a3e-3146-4ecd-8a56-64764117e8da" />

here's what I see from OpenInference

<img width="893" height="293" alt="Image" src="https://github.com/user-attachments/assets/dee41518-835d-445b-93c7-78f334932a32" />

-----
TL;DR:

It seems we're missing agent-level spans in workflow scenarios, I'd like to make sure we didn't introduce something here that broke it or would prevent from having them in the future

Comment thread instrumentation-genai/opentelemetry-instrumentation-langchain/pyproject.toml Outdated
run_id, parent_run_id, None
)

def on_chain_end(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any content attributes on the workflow span (input/output/instructions) - is it intentional?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added input and output messages on workflow and invoke agent spans,

invoke_workflow
Screenshot 2026-05-11 at 4 26 54 PM

invoke_agent
Screenshot 2026-05-11 at 4 51 34 PM

@wrisa

wrisa commented May 11, 2026

Copy link
Copy Markdown
Contributor Author

Not sure if we should expect to see invoke_agent somewhere?

The nodes draft, critique, revise, finalize are plain Python functions registered as LangGraph nodes. When LangGraph calls on_chain_start for each of them, the metadata it injects contains langgraph_node: "draft" etc., but none of the agent signals the classifier looks for:

  def _has_agent_signals(metadata):                                                                                                                                                                                                                 
      return bool(
          metadata.get("otel_agent_span")   # not set
          or metadata.get("agent_name")     # not set
          or metadata.get("agent_type")     # not set
      )

LangGraph doesn't mark its nodes as agents — it's just a graph of Python functions. Hence, there will not be invoke_agent span.
See agent_name added as metadata in this example

@lmolkova

Copy link
Copy Markdown
Member

LangGraph doesn't mark its nodes as agents — it's just a graph of Python functions. Hence, there will not be invoke_agent span.

Can we discuss it? It seems logical that individual nodes become agents and orchestration over them becomes a workflow. It also aligns with OpenInference and Langfuse if I read this image right.

@lmolkova lmolkova left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wrisa could you please more this PR to https://github.com/open-telemetry/opentelemetry-python-genai and let's continue the discussion there? Sorry for the inconvenience.

@github-project-automation github-project-automation Bot moved this from Approved PRs to Reviewed PRs that need fixes in Python PR digest May 13, 2026
@wrisa

wrisa commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

It also aligns with OpenInference and Langfuse if I read this image right.
@lmolkova
OpenInference (openinference-instrumentation-langchain)

  Key logic (_tracer.py, lines 288–292):
  span_kind = (
      OpenInferenceSpanKindValues.AGENT
      if "agent" in run.name.lower()
      else _langchain_run_type_to_span_kind(run.run_type)
  )
  • It sets span kind to AGENT if the run name contains the string "agent" (case-insensitive) — e.g., math_agent, AgentExecutor.
  • Otherwise it falls back to the LangChain run type (chain, llm, tool, etc.).
  • A plain LangGraph node named researcher would get span_kind = CHAIN, not AGENT.

Langfuse (langfuse-python)

  Key logic (CallbackHandler.py, lines 393–404):
  elif callback_type == "chain":
      if serialized and "id" in serialized:
          class_path = serialized["id"]
          if any("agent" in part.lower() for part in class_path):
              return "agent"
      name = self.get_langchain_run_name(serialized, **kwargs)
      if "agent" in name.lower():
          return "agent"
      return "chain"
  • Maps to Langfuse's internal observation types: "agent", "chain", "generation", "tool", "span".
  • A researcher node returns "chain", not "agent".

Both actually apply the same name-based heuristic as the current implementation — they only mark something as "agent" if the word "agent" appears in the run name or class path.
see eg: workflow. A plain LangGraph node named researcher or summariser would be classified as chain in both systems, not agent.

Created issue for further discussion.

@wrisa

wrisa commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

@wrisa could you please more this PR to https://github.com/open-telemetry/opentelemetry-python-genai and let's continue the discussion there? Sorry for the inconvenience.

open-telemetry/opentelemetry-python-genai#25

@lzchen

lzchen commented May 18, 2026

Copy link
Copy Markdown
Contributor

Closing in favor of open-telemetry/opentelemetry-python-genai#25

@lzchen lzchen closed this May 18, 2026
@github-project-automation github-project-automation Bot moved this from Reviewed PRs that need fixes to Done in Python PR digest May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gen-ai Related to generative AI

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants